Moving picture decoding apparatus using replacement image data

ABSTRACT

A moving picture decoding apparatus is provided which reduces the deterioration in image quality due to errors by a concealment processing for decoded image data, thereby improving the image quality of decoded images, without causing high deterioration in image quality resulting from the concealment processing. The moving picture decoding apparatus includes a decoder for decoding an input stream for each macroblock and generating decoded image data, a transmission error detector for detecting a transmission error in the input stream, and a stream error detector for detecting a stream error in the input stream. When the transmission error is detected, the moving picture decoding apparatus conceals the decoded image data in macroblock units and when the stream error is detected, the moving picture decoding apparatus conceals the decoded image data in video packet units.

This application is a divisional of U.S. application Ser. No.09/667,241, filed Sep. 22, 2000 now U.S. Pat. No. 6,999,673.

FIELD OF THE INVENTION

The present invention relates to a moving picture decoding method, amoving picture decoding apparatus, and a recording medium which containsa program. More particularly, it relates to a process of concealingdecoded image data which are obtained by decoding a bitstream whichincludes an error, so as to obtain visually preferable decoded images.

BACKGROUND OF THE INVENTION

In recent years, we entered the age of multimedia in which voices,images and other representative media are integratedly operated, andmeans for transmitting information of conventional information media,such as newspapers, magazines, television, radio and telephone, topersons have been adopted as targets of the multimedia. The multimediagenerally means representing not only characters but also drawings,voices, particularly images and the like, which are simultaneouslycorrelated with the characters. However, in order to make theconventional information media the targets of the multimedia, it isindispensable to represent the information in digital formats.

Estimating the amount of the information in each of the informationmedia as a digital information amount, the information amount ofcharacters (per character) is 1˜2 bytes, the information amount ofvoices of telephone quality is 64 Kbits per second, and the informationamount of moving pictures of the current television receiving quality is100 Mbits per second. Accordingly, in information media such as thetelephone and television, it is not practical that massive informationin digital formats is operated as it is. For example, TV phones havealready been put to practical use by ISDN (Integrated Service DigitalNetwork) having the transmission rate of 64 kbps˜1.5 Mbps. However,video information which is obtained by TV cameras cannot be transmittedas digital data as it is by the ISDN.

Thus, technology for compressing information is required. For example,in the case of TV phones, the moving picture compression technologyaccording to H.261 and H.263 standards which are internationallystandardized by ITU-T (International TelecommunicationUnion-Telecommunication Sector) is employed. In addition, according tothe information compression technology of MPEG-1 standard, imageinformation can be stored in normal music CDs (Compact Disks) togetherwith voice information.

Here, MPEG (Moving Picture Experts Group) is the international standardrelating to the processing for compressing moving picture data. MPEG-1is the standard for compressing moving picture data in up to 1.5 Mbps,i.e., compressing information of television signals by about onehundredth. Further, the transmission rate for MPEG-1 standard isprincipally restricted to about 1.5 Mbps. The moving picture data arecompressed into 2˜15 Mbps according to MPEG-2 which is standardized tomeet requirements of higher image quality.

Further, in the present circumstances, MPEG-4 which enables coding ofimage data and operation of image data in object units, and realizes newfunctions required in the age of multimedia is now being standardized bya working group (ISO/IEC JTC1/SC29/WG11) that has promoted thestandardization of the compression processing for moving picture data,like MPEG-1 and MPEG-2. In MPEG-4, the standardization of the codingprocessing at a lower bit rate has been aimed at first, but at thepresent time the targets of the standardization are extended togeneral-purpose coding processing for interlaced images at a higher bitrate.

One of characteristics of MPEG-4 is a mechanism for simultaneouslycoding image data corresponding to plural image sequences (i.e., pluralmoving pictures) and transmitting them. This mechanism makes it possibleto construct one scene by composing plural images. The image in thiscase is an image (still picture) of each frame of the image sequence(moving picture). One scene is a composed image including plural images.

For example, in MPEG-4, it is possible that the foreground andbackground constituting one scene are separated as images (objects) ofdifferent image sequences and that the frame frequency, image qualityand bit rate are changed independently for each of the image sequences.In addition, in MPEG-4, the images of the plural image sequences arearranged in the horizontal or vertical direction, like multi-screens,whereby users can extract or enlarge to display only images of desiredimage sequences.

As for the background, the coding processing only for pixel valuesignals (texture signals) indicating the brightness and tint isgenerally performed, as in MPEG-2. On the other hand, as for theforeground, the processing of not only coding the pixel value signalsindicating the brightness and tint of the objects but alsosimultaneously coding shape signals indicating shapes of the objects isperformed. Generally, this coding processing for the foreground is knownas coding processing in object units.

According to MPEG-4, the whole of a displayed image (composed image) iscomposed of images (objects) of plural image sequences. Therefore, aframe of each image sequence at each display time is referred to as aVOP (Video Object Plane) and distinguished from frames according toMPEG-1 and 2. When the whole of the displayed image is composed ofimages of one image sequence, the VOP and the frame coincide.

FIGS. 8( a) to 8(c) are diagrams for schematically explaining codingprocessing in object units according to MPEG-4.

An image signal specified according to MPEG-4 is composed of a shapesignal representing the shape Sob of an object (VOP) (FIG. 8( a)) and apixel value signal (texture signal) comprising a brightness signal and acolor difference signal, representing the texture Tob of the object(VOP) (FIG. 8( b)).

In the coding processing in object units, it is necessary to decide theshape of the object and the position of the object with respect to areference coordinate system for displaying the image. Thus, arectangular region (bounding box) Box (FIG. 8( c)) is composed of pluralmacroblocks, involving the object Ob, is decided by the referencecoordinate system. Here, the macroblock is an image space as a unit ofthe coding processing, and composed of 16×16 pixels. In addition, sincethe rectangular region Box is composed of the plural macroblocks, thenumber of pixels in the horizontal and vertical directions in therectangular region is a multiple of 16.

Then, each of the rectangular regions Box in one image sequence issubjected to the coding processing of coding the image signal for eachmacroblock.

For example, in FIG. 8( c), the rectangular region Box is composed of5×4 macroblocks. Macroblocks MB1 and MB2 are macroblocks situatedoutside the object Ob (macroblock-outside-object). A macroblock MB3 is aboundary macroblock situated on the boundaries of the object Ob. Amacroblock MB13 is a macroblock situated inside the object Ob(macroblock-inside-object). According to MPEG-4, pixels outside theobject are not displayed after the decoding. Therefore, the codingprocessing is performed only for macroblocks including the pixels insidethe object, which are displayed after the decoding, i.e., the boundarymacroblocks and macroblocks-inside-object.

FIGS. 9( a) to 9(e) are diagrams for schematically explaining variousprocessing units in a bitstream which conforms to MPEG-4.

Here, a rectangular region (bounding box) Box including an object (VOP)corresponds to the object in a one-to-one relationship. Therefore, inthe following description, the rectangular region (bounding box) Box andthe object (VOP) are not distinguished from each other and they arereferred to as VOPs.

Generally, in a code sequence (bitstream) composed of variable lengthcodes, fixed length codes comprising specific bit patterns are arrangedto prevent the error propagation at the decoding time. According toMPEG-4, this fixed length code is referred to as Resync Marker(hereinafter abbreviated as only marker) and it is a synchronizationsignal. A code sequence composed of this marker and the variable lengthcodes subsequent to the marker is a coding unit which is referred to asa video packet.

According to MPEG-4, as shown in FIG. 9( a), a code sequence (VOPbitstream) Svop corresponding to one VOP 10 can be composed of pluralvideo packets. In this case, the VOP bitstream Svop is composed of fourvideo packets Svp1˜Svp4. Coded data corresponding to respective regionsRvp1˜Rvp4 in the VOP 10 are stored in the video packets Svp1˜Svp4,respectively. In addition, coded data corresponding to pluralmacroblocks can be stored in each of the video packets Svp1˜Svp4.

Here, the region Rvp1 corresponding to the video packet Svp1 is composedof five macroblocks MB1˜MB5 as shown in FIG. 9( b). Also each of theregions Rvp2˜Rvp4 corresponding to the other video packets Svp2˜Svp4 iscomposed of five macroblocks as the region Rvp1 corresponding to thevideo packet Svp1. In addition, each of the macroblocks is an imagespace composed of 16 pixels×16 pixels as described above and composed offour blocks. Each of the blocks is an image space composed of 8 pixels×8pixels. For example, the macroblock MB1 is composed of blocks B1˜B4 asshown in FIG. 9( c), and the block B1 is composed of 8 pixels×8 pixelsas shown in FIG. 9( d).

In addition, the coded data corresponding to one macroblock (hereinafterreferred to also as macroblock information) include brightnessinformation (Y) which corresponds to the four blocks constituting onemacroblock, and color difference information (U) and (V) correspondingto one macroblock. Further, when the object has the shape, the codeddata of one macroblock include shape information corresponding to onemacroblock, together with the brightness information (Y) and the colordifference information (U) and (V).

Here, the brightness information (Y) of one macroblock is obtained bycoding the pixel value signals of the four blocks constituting onemacroblock. The color difference information (U) and (V) of onemacroblock is obtained by coding the color difference signals (U) and(V) of 8 pixels×8 pixels constituting one macroblock, respectively. Theshape information of one macroblock is obtained by coding the shapesignal of 16 pixels×16 pixels constituting one macroblock.

It is unnecessary that the number of the macroblocks constituting theregion corresponding to the video packet in the VOP 10 is always fixedas shown in FIG. 9( a). For example, as shown in FIG. 9( e), the numbersof macroblocks constituting regions Rvp1 a˜Rvp5 a which correspond tovideo packets in a VOP 10 a may be decided so that the amounts of codesin the respective video packets Svp1 a˜Svp5 a in a VOP bitstream Svopaare fixed. In this case, the numbers of the macroblocks included in theregions Rvp1 a˜Rvp5 a corresponding to the respective video packets arenot fixed.

FIG. 10 is a diagram for schematically explaining the coding processingaccording to MPEG-4 in object units. This figure shows the codingprocessing for the image signal corresponding to the object (VOP) havingthe shape as shown in FIG. 8.

Here, the VOP 10 as the object Ob (strictly speaking Bounding Box BBOXincluding the object) is composed of four video packet regions Rvp1˜Rvp4each composed of five macroblocks. For example, the video packet regionRvp1 is composed of the macroblocks MB1˜MB5.

The macroblocks MB1 and MB2 are situated outside the object. Therefore,these macroblocks MB1 and MB2 are subjected to the processing of codingthe shape signal indicating that the macroblocks are outside the objectas the coding processing for the shape signal, and the processing ofcoding the pixel value signal is omitted. In addition, the macroblockMB3 is subjected to the coding processing for the shape signal and thecoding processing for the pixel value signal, because this macroblock isa macroblock including pixels inside the object.

Generally, many objects having the shapes as the foreground have theshapes or sizes varying from moment to moment, unlike objects as thebackground. In addition, according to MPEG-4, the coding algorithm ofthe shape signals or pixel value signals greatly depends on the shapesof the images to be coded. For example, when an object has the shape,the coding processing of the pixel value signal is omitted for a part(macroblock) which is indicated by the shape signal that it is outsidethe object. Therefore, there are some cases where the number of themacroblocks which are subjected to the coding processing of the pixelvalue signals, corresponding to one scene (VOP) of one image sequencemay vary. Accordingly, the decoding processing conforming to MPEG-4 iseasily affected by transmission errors of the bitstream, as comparedwith the decoding processing for the coding processing of images havingthe shapes and sizes which do not vary like MPEG-2. Further, in thisdecoding processing, the image concealment such as the image restorationor image processing utilizing the correlation between VOPs is alsodifficult. Consequently, in the decoding system for MPEG-4, when thetransmission error occurs, the image quality in a decoded image isconsiderably deteriorated.

FIGS. 11( a) to 11(c) are diagrams for schematically explaining thestructure of a bitstream which conforms to MPEG-4 in detail.

The VOP bitstream Svop includes coded data corresponding to the VOP 10as the object Ob as shown in FIG. 10. At the head of the VOP bitstreamSvop, a VOP header Svoph as important data relating to the whole VOP isarranged. The video packets Svp1˜Svp4 are arranged subsequent to the VOPheader Svoph (see FIG. 11( a)).

In the video packet Svp1, a video packet header Svph as important datarelating to the whole video packet is arranged at its head. Coded data(macroblock information) Smb1˜Smb5 corresponding to the macroblocksMB1˜MB5 are arranged subsequent to the video packet header Svph (seeFIG. 11( b)).

Further, a macroblock header Smbh as important data relating to thewhole macroblock is arranged at the head of the macroblock informationSmb1. Shape information Ssb, of the corresponding macroblock, brightnessinformation Spb1˜Spb4 of four blocks constituting the correspondingmacroblock, and color difference information (U) Spbu and (V) Spbv ofthe corresponding macroblock are arranged subsequent to the macroblockheader Smbh (see FIG. 11( c)).

Thus, in the VOP bitstream Svop, the macroblock informationcorresponding to the macroblock as the coding unit is the firstprocessing unit. Further, the video packet composed of the plural piecesof macroblock information is the second processing unit. The VOPbitstream has a two-layer data structure in which the coded dataincluded therein are divided by the first and second processing units.

Here, the VOP header Svoph and the video packet header Svph includesynchronization signals for synchronizing the decoding processing of thebitstream. Thus, when the decoding processing of the bitstreams isinterrupted due to an error bit in the bitstream, the decodingprocessing can be resumed from the VOP header Svoph or video packetheader Svph. On the other hand, the macroblock header Smbh includes nosynchronization signal for synchronizing the decoding processing. Here,the synchronization signal in the video packet header Svph is theabove-mentioned fixed length code (Resync Marker).

Generally, as errors in the bitstream in the moving picture decodingprocessing, there are two kinds of errors, i.e., a stream error and atransmission error.

As the stream error, there is an error that a code which isgrammatically incorrect is included in a stream (syntax error) or anerror that a code of an incorrect value which exceeds a range ofavailable values is included (semantic error) and the like. Thetransmission error is an error in which the bitstream is destroyed whenthe bitstream is read from a recording medium or the bitstream istransmitted via a communication medium due to the missing of data or thelike.

Usually, the coded image data corresponding to each VOP are stored in atransmission packet having header information, and transmitted as theVOP bitstream in transmission packet units. Therefore, when thetransmission error such as the absence of packets occurs, the positionof a missing transmission packet in the bitstream can be detected on thereceiving end. Accordingly, as for the transmission error, the positionwhere the decoding processing fails in the bitstream (error occurrenceposition) can be almost specified.

As a concrete method for specifying the error occurrence position in thedecoding processing, there is a method of detecting the absence ofpackets in the bitstream and adding a mark which indicates the absenceof the packet (marker code) to the position of the missing packet in thebitstream.

Compared with this transmission error, the stream error results from thesyntax error occurring at the variable-length coding time or the like.Therefore, this error cannot be detected as a decoding error until thedecoding process such as the variable-length decoding processing fails.In other words, the stream errors cannot be detected essentially unlessthe decoding process of the bitstream (coded data) fails.

However, the synchronization signal is arranged at the head of one videopacket. In addition, immediately after this video packet, thesynchronization signal of the subsequent video packet is arranged.Therefore, when the structure and contents of the bitstream situatedbetween these two synchronization signals are strictly examined in thedecoding process, only video packets including the stream errors can bedetected regardless of the failure of the decoding process of thebitstream. When the structure and contents of the bitstream are strictlyexamined in the decoding process in this way, the possibility ofdetecting the stream errors is considerably higher as compared with thecase where the failure of the decoding process of the bitstream isdetected.

Hereinafter, a conventional moving picture decoding apparatus will bedescribed in detail.

FIG. 12 is a block diagram for explaining the conventional common movingpicture decoding apparatus.

This moving picture decoding apparatus 100 receives a bitstream readfrom a recording medium or a bitstream transmitted via a transmissionmedium as an input stream Vin and performs decoding processing for theinput stream Vin. Here, the bitstream includes image coded data whichare obtained by subjecting an image signal of a moving picture to codingprocessing separately for each of plural image sequences constitutingthe moving picture. In addition, the coding processing for the imagesignal of one of the image sequences is performed for each scene (VOP)of the image sequence and the image signal corresponding to each VOP iscoded in units of macroblocks constituting the VOP. It goes withoutsaying that the image signal of the object having no shape includes onlythe brightness signal and the color difference signal, and the imagesignal of the object having the shape includes the shape signal togetherwith the brightness signal and the color difference signal.

The bitstream corresponding to the moving picture normally includes theimage coded data corresponding to each object being multiplexed.However, in the following description, assume that the bitstreamincludes only image coded data corresponding to one object as imageinformation.

To be specific, the moving picture decoding apparatus 100 includes adecoder 101 for performing the decoding processing of the input streamVin corresponding to a target VOP to be processed for each macroblockwith reference to decoded image data (reference image data) Vref in areference region in an already processed VOP whose decoding processingis finished, and outputting decoded image data Vd, and a memory 102 forsynchronizing the reference image data Vref with decoding processing ofa macroblock to be processed (target macroblock) in the target VOP, andoutputting the data as well as synchronizing decoded image data(replacement image data) Vrep corresponding to a macroblock in thealready processed VOP whose relative position in the already processedVOP is equal to the relative position of the target macroblock in thetarget VOP with the decoding processing for the target macroblock andoutputting the data.

Further, the moving picture decoding apparatus 100 includes an errordetector 120 for detecting an error of the input stream Vin and itsposition on the basis of the input stream Vin and outputting an errornotification signal Terr, a selector switch 105 for selecting one of thedecoded image data Vd of the target macroblock and the replacement imagedata Vrep in accordance with a control signal Cmb, and outputting theselected image data (MB selected image data) Emb as reproduced imagedata Vout of the target macroblock, and a macroblock unit concealer 104for generating the control signal Cmb for the selector switch 105 inaccordance with the error notification signal Terr.

Here, the error detector 120 detects the error in the input stream Vinusing the level of the input stream Vin as an analog signal or anerror-correcting signal included in the input stream. Therefore, in thiserror detector 120, the transmission errors are detected.

The macroblock unit concealer 104 controls the selector switch 105 inaccordance with the error notification signal Terr so that in place ofthe decoded image data Vd which are obtained by decoding the macroblockinformation from the macroblock information including the error part ofthe input stream Vin to the subsequent synchronization signal, thedecoded image data (replacement image data) Vrep of the alreadyprocessed VOP corresponding to the decoded image data are output as thereproduced image data Vout.

Next, the operation of the moving picture decoding apparatus 100 isdescribed.

When a bitstream read from the recording medium or a bitstreamtransmitted via the transmission medium is input to the moving picturedecoding apparatus 100 as the input stream Vin, the decoding processingfor the input stream is performed in macroblock units for each VOP inthe moving picture decoding apparatus 100. Here, in this moving picturedecoding apparatus 100, the decoder 101, the memory 102 and themacroblock unit concealer 104 are controlled by a control unit (notshown) in this apparatus 100 during the decoding processing so that theprocessings for the respective macroblocks are synchronized betweenthese units.

To be specific, in the decoder 101, the decoding processing is performedfor the coded data of the target macroblock in the target VOP, withreference to the reference image data Vref corresponding to the targetmacroblock, and the decoded image data Vd of the target macroblock areoutput. When the input stream Vin includes an error, the decoder 101outputs only the decoded image data Vd corresponding to the macroblockwhose coded data can be decoded.

At this time, the memory 102 outputs the replacement image data Vrepcorresponding to the target macroblock together with the reference imagedata Vref corresponding to the target macroblock.

The error detector 120 performs error detection processing of detectingtransmission errors on the basis of the input stream Vin. When the errorof the input stream is detected, the error detector 120 outputs an errornotification signal Terr which shows the macroblock informationincluding an error part of the input stream as the position of the errorpart, to the macroblock unit concealer 104.

Then, the macroblock unit concealer 104 outputs the control signal Cmbto the selector switch 105 for selecting one of the decoded image dataVd of the target macroblock and the replacement image data Vrepcorresponding to the target macroblock, in accordance with the errornotification signal Terr. That is, the selector switch 105 is controlledso that the replacement image data Vrep from the memory 102 areselected, in place of the decoded image data Vd from the decoder 101,for the macroblock corresponding to each of macroblock information fromthe macroblock information indicated by the error notification signalTerr to the subsequent synchronization signal, and the decoded imagedata Vd output by the decoder 101 are selected for other macroblocks.

Then, the selected image data Emb which are selected by the selectorswitch 105 are output as the reproduced image data Vout corresponding tothe target macroblock of the target VOP. In addition, the selected imagedata Emb are recorded in the memory 102 as the reference image data fora VOP subsequent to the target VOP.

At this time, not only the decoded image data of the error macroblock(macroblock whose macroblock information includes the error part of thebitstream) but also the decoded image data of all macroblocks subsequentto the error macroblock in the video packet are replaced with thedecoded image data (replacement image data) Vrep of the correspondingmacroblocks in the already processed VOP, because the input stream isobtained by the variable-length coding processing for the image data. Tobe specific, in the variable-length decoding processing for the inputstream, when the input stream includes an error, the error affects thedecoding processing for all of macroblock information from the erroroccurrence position in the input stream to the synchronization signal.

FIG. 13 is a block diagram for explaining another conventional movingpicture decoding apparatus.

This moving picture decoding apparatus 110 conceals the decoded imagedata which are obtained by the decoding processing for the input streamwhich includes errors, not in macroblock units as in the moving picturedecoding apparatus 100 but in video packet units.

To be specific, this moving picture decoding apparatus 110, like themoving picture decoding apparatus 100 shown in FIG. 12, has a decoder101 for performing the decoding processing for the input stream Vincorresponding to the target VOP with reference to the reference imagedata Vref and outputting the decoded image data Vd corresponding to eachmacroblock, and a memory 102 for synchronizing the reference image dataVref and the replacement image data Vrep for the target macroblock withthe decoding processing of the target macroblock and outputting thedata.

Further, in place of the selector switch 105 in the moving picturedecoding apparatus 100, the moving picture decoding apparatus 110 has afirst delay circuit 103 for delaying the decoded image data Vd for atime required for the decoding processing of a target video packet to beprocessed, a second delay circuit 104 for delaying the replacement imagedata Vrep which are output by the memory 102 in synchronization with thedecoding processing of each macroblock, for a time required for thedecoding processing of the target video packet, and a selector switch108 for selecting one of the output (delayed decoded data) DVd of thefirst delay circuit 103 and the output (delayed replacement data) DVrepof the second delay circuit 104, in accordance with a control signalCvp.

Further, in place of the error detector 120 in the moving picturedecoding apparatus 100, this moving picture decoding apparatus 110 hasan error detector 121 for detecting the failure of the normal decodingprocessing in the decoder 101 in accordance with an internal signal Siof the decoder, and outputting an error notification signal Nerrindicating the error detection. This error detector 121 can have astructure for detecting the abnormality of the bitstream in the videopacket by the processing of

strictly examining the structure and contents of the bitstream, in placeof the processing of detecting the failure of the normal decodingprocessing, and outputting the error notification signal Nerr indicatingthe error detection.

Further, in place of the macroblock unit concealer 104 in the movingpicture decoding apparatus 100, this moving picture decoding apparatus110 has a video packet unit concealer 107 for controlling the selectorswitch 108 to select one of the delayed decoded data DVd from the firstdelay circuit 103 and the delayed replacement data DVrep from the seconddelay circuit 104 for each macroblock, in accordance with the errornotification signal Nerr.

To be specific, this video packet unit concealer 107 controls theselector switch 108 in accordance with the error notification signalNerr so as to output, in place of the delayed decoded data DVdcorresponding to a video packet (error video packet) whose decodingprocessing in the decoder 101 fails, the delayed replacement data DVrepof a video packet of the already processed VOP corresponding to theerror video packet as the reproduced image data Vout.

Here, the decoder 101 and the memory 102 in the moving picture decodingapparatus 110 as shown in FIG. 13 have the same structures as those ofthe decoder 101 and the memory 102 in the moving picture decodingapparatus 100 as shown in FIG. 12.

Next, the operation of the moving picture decoding apparatus 110 isdescribed.

In this moving picture decoding apparatus 110, the decoding processingfor the input stream Vin in the decoder 101, and outputting of thereference image data Vref and the replacement image data Vrep from thememory 102 is performed in the same way as in the moving picturedecoding apparatus 100.

In this moving picture decoding apparatus 110, the decoded image data Vdfrom the decoder 101 are delayed by the first delay circuit 103 for atime required for the decoding processing for a target video packet tobe decoded, and the replacement image data Vrep from the memory 102 aredelayed by the second delay circuit 104 for a time required for thedecoding processing for the target video packet.

In the error detector 121, the processing of detecting the failure ofthe decoding processing for the input stream is performed in accordancewith the internal signal Si in the decoder 101. When the failure of thedecoding processing is detected, the error notification signal Nerrindicating the detection of the error is output to the video packet unitconcealer 107. This video packet unit concealer 107 outputs the controlsignal Cvp to the selector switch 108 in accordance with the errornotification signal Nerr. The selector switch 108 selects one of thedelayed decoded data DVd from the first delay circuit 103 and thedelayed replacement data DVrep from the second delay circuit 104 inaccordance with the control signal Cvp, and outputs the selected data(VP unit selected data) Evp as the reproduced image data Vout.

To be more specific, the selector switch 108 is controlled by the videopacket unit concealer 107 so that, in place of the delayed decoded dataDVd corresponding to the video packet in which the error is detected(error video packet), the delayed replacement data DVrep of the videopacket of the already processed VOP, corresponding to the error videopacket, are output as the reproduced image data Vout.

The reproduced image data Vout of the target VOP are recorded in thememory 102 as the reference image data for the VOP subsequent to thetarget VOP.

The so-constructed moving picture decoding apparatus 110 detects thefailure of the decoding processing, and replaces the decoded image dataVd of the video packet whose decoding processing fails, with the decodedimage data of the corresponding video packet in the already processedVOP. The decoding processing normally fails when the input streamincludes errors. Therefore, when the bitstream including thetransmission errors or stream errors is input, the concealment of thedecoded image data is performed.

However, the above-mentioned conventional moving picture decodingapparatus, i.e., the conventional moving picture decoding apparatus 100which conceals the decoded image data in macroblock units (see FIG. 12)and the conventional moving picture decoding apparatus 110 whichconceals the decoded image data in video packet units (see FIG. 13) havefollowing problems, respectively.

To be specific, the moving picture decoding apparatus 100 as shown inFIG. 12 detects errors using the analog signal level of the input streamor the error-correcting codes, and conceals the decoded image data inmacroblock units, whereby the decoded image data can be concealedcarefully. However, the stream errors cannot be detected using theanalog signal level of the input stream or the error-correcting codes.Therefore, the deterioration in image quality of the decoded image dueto the stream errors cannot be reduced.

The moving picture decoding apparatus 110 as shown in FIG. 13 detectserrors on the basis of the occurrence of the failure in the decodingprocessing, and conceals the decoded image data in video packet units.Therefore, the decoded image data corresponding to the normal macroblockinformation including no error part, precedent to the error macroblockare also replaced with the decoded image data of the already processedVOP. Thus, the image quality of the decoded image is considerablydeteriorated due to the concealment of the decoded image data andthereby the concealment of the transmission errors or stream errors inthe decoded image cannot be performed effectively.

Therefore, conventionally, the moving picture decoding apparatus whichconceals the decoded image in macroblock units as shown in FIG. 12 andthe moving picture decoding apparatus which conceals the decoded imagein video packet units as shown in FIG. 13 are used properly according totheir purposes.

Further, both of the above-mentioned conventional moving picturedecoding apparatus have the structures in which the concealmentprocessing for the decoded image is performed without distinctionbetween the case where the input stream has the shape information andthe case where the input stream has no shape information. Therefore, inthe case where the input stream has the shape information, a good imagequality is not obtained even when the concealment of the image isperformed.

That is, there are many cases where objects as the targets of the codingprocessing in object units have the shapes which considerably vary frommoment to moment. Therefore, when the image of a part of the target VOPis concealed utilizing an image in the already processed VOP, thecontinuity in shape within the target VOP is often harmed betweenconcealed parts and unconcealed parts in the target VOP. When thecontinuity in shape is harmed, the concealed parts show and the imagequality is substantially deteriorated.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a moving picturedecoding method and a moving picture decoding apparatus, which caneffectively improve image quality deteriorated due to transmissionerrors or stream errors in decoded images which are obtained by thedecoding processing for input streams by concealment processing for thedecoded images, and a recording medium which contains a moving picturedecoding program for implementing the moving picture decoding method bysoftware.

According to a 1st aspect of the present invention, there is provided amoving picture decoding method of subjecting a bitstream including codeddata which are obtained by successively coding image data correspondingto a moving picture for each of first processing units andsynchronization signals each of which is added to the coded data foreach second processing unit including a plurality of the firstprocessing units, to decoding processing of decoding the coded data foreach of the first processing units and generating decoded image data,comprising: an error detection process of detecting a first erroroccurring during transmission of the bitstream and a second error otherthan the first error separately; and a concealment process of performingconcealment of the decoded image data taking the first processing unitas a concealment unit when the first error is detected, and performingconcealment of the decoded image data taking the second processing unitas a concealment unit when the second error is detected. Therefore, theconcealment of the decoded image data for both of the transmission errorand the stream error can be performed.

In addition, when the transmission error occurs, only decoded image datacorresponding to the first processing unit which are affected by thetransmission error are concealed. Therefore, the deterioration in imagequality of decoded images due to the concealment of decoded image datacorresponding to the first processing unit which are not affected by thetransmission error can be avoided. Further, when the stream error otherthan the transmission error occurs, decoded image data corresponding tothe second processing unit having a synchronization signal attachedthereto are concealed. Therefore, the deterioration in image quality ofdecoded images due to output of the decoded image data corresponding tothe first processing unit which are affected by the stream error can beprevented.

Consequently, when the error in the input bitstream is detected, theconcealment processing for the decoded image data is performedeffectively, whereby the deterioration in image quality of the decodedimages can be considerably reduced.

According to a 2nd aspect of the present invention, there is provided amoving picture decoding method of subjecting a bitstream including codeddata which are obtained by successively coding image data correspondingto a moving picture for each of unit regions constituting one VOP (VideoObject Plane) of the moving picture, to decoding processing of decodingthe coded data for each of the unit regions and generating decoded imagedata, comprising: an error detection process of detecting an error inthe bitstream; a process of judging whether or not the bitstream hasshape information indicating a shape of the moving picture; and aconcealment process of performing concealment of the decoded image datataking one VOP of the moving picture as a concealment unit when theerror in the bitstream is detected and it is judged that the bitstreamhas the shape information, and performing concealment of the decodedimage data taking a processing region including at least one of the unitregions, which processing region is smaller than the VOP, as aconcealment unit when the error in the bitstream is detected and it isjudged that the bitstream has no shape information. Therefore, when thebitstream includes the shape information, the deterioration in imagequality resulting from the error can be excluded without causing highdeterioration in image quality due to the concealment processing for thedecoded images. Moreover, when the stream has no shape information, thedeterioration in image quality caused by the error can be excluded bythe simple concealment processing.

According to a 3rd aspect of the present invention, in the movingpicture decoding method of the 2nd aspect, the bitstream includessynchronization signals each of which is added to the coded data foreach second processing unit which includes a plurality of firstprocessing units each corresponding to the unit region, the errordetection process is a process of detecting a first error occurringduring transmission of the bitstream and a second error other than thefirst error separately, and the concealment process is a process ofperforming the concealment of the decoded image data taking the firstprocessing unit as the concealment unit when the first error in thebitstream having no shape information is detected, and performing theconcealment of the decoded image data taking the second processing unitas the concealment unit when the second error in the bitstream having noshape information is detected. Therefore, the deterioration in imagequality in decoded images which are obtained by the decoding processingfor the bitstream having shape information due to the transmission erroror stream error can be excluded with suppressing the deterioration inimage quality resulting from the concealment of the decoded image data.Besides, when the bitstream has no shape information, the concealmentprocessing for the error of the decoded image data is performedeffectively according to the types of the error, whereby the imagequality of the decoded images can be improved.

According to a 4th aspect of the present invention, there is provided amoving picture decoding apparatus which subjects a bitstream includingcoded data which are obtained by successively coding image datacorresponding to a moving picture for each of first processing units andsynchronization signals each of which is added to the coded data foreach second processing unit which includes a plurality of the firstprocessing units, to decoding processing, comprising: a decoder fordecoding the coded data included in the bitstream for each of the firstprocessing units and generating decoded image data; a first errordetector for detecting a first error occurring during transmission ofthe bitstream; a second error detector for detecting a second errorother than the first error; a first concealer for performing concealmentof the decoded image data taking the first processing unit as aconcealment unit when the first error is detected; and a secondconcealer for performing concealment of the decoded image data takingthe second processing unit as a concealment unit when the second erroris detected. Therefore, the concealment of the decoded image data can beperformed in appropriate processing units according to whether the erroris the transmission error or stream error, in the same way as in themoving picture decoding method according to the 1st aspect of thepresent invention. Accordingly, when the error in the input bitstream isdetected, the concealment processing for the decoded image data isperformed effectively, whereby the deterioration in image quality ofdecoded images is considerably reduced.

According to a 5th aspect of the present invention, there is provided amoving picture decoding apparatus which subjects a bitstream includingcoded data which are obtained by successively coding image datacorresponding to a moving picture for each of unit regions constitutingone VOP of the moving picture, to decoding processing, comprising: adecoder for decoding the coded data included in the bitstream for eachof the unit regions and generating decoded image data; an error detectorfor detecting an error in the bitstream; a unit for judging whether ornot the bitstream has shape information indicating a shape of the movingpicture; and a decoded image concealer for performing concealment of thedecoded image data taking one VOP of the moving picture as a concealmentunit when the error in the bitstream is detected and it is judged thatthe bitstream has the shape information, and performing concealment ofthe decoded image data taking a processing region including at least oneof the unit regions, which processing region is smaller than the VOP, asa concealment unit when the error in the bitstream is detected and it isjudged that the bitstream has no shape information. Therefore, when thestream includes the shape information, the deterioration in imagequality resulting from the error can be excluded without causing highdeterioration in image quality due to the concealment processing for thedecoded images. Besides, when the stream has no shape information, thedeterioration in image quality due to the error can be excluded by thesimple concealment processing.

According to a 6th aspect, in the moving picture decoding apparatus ofthe 5th aspect, the bitstream includes synchronization signals each ofwhich is added to the coded data for each second processing unit whichincludes a plurality of first processing units each corresponding to theunit region, the error detector comprises: a first error detector fordetecting a first error occurring during transmission of the bitstream;and a second error detector for detecting a second error other than thefirst error, and the decoded image concealer comprises: a firstconcealer for performing the concealment of the decoded image datataking the first processing unit as the concealment unit when the firsterror in the bitstream having no shape information is detected; a secondconcealer for performing the concealment of the decoded image datataking the second processing unit as the concealment unit when thesecond error in the bitstream having no shape information is detected;and a third concealer for performing the concealment of the decodedimage data taking one VOP of the moving picture as the concealment unitwhen one of the first error and the second error in the bitstream havingthe shape information is detected. Therefore, the deterioration in imagequality of the decoded image having the shape due to the transmissionerror or stream error can be excluded with suppressing the deteriorationin image quality resulting from the concealment of the decoded imagedata. Besides, when the bitstream has no shape information, theconcealment processing for the error of the decoded image data can beperformed effectively according to the types of error, whereby the imagequality of the decoded image is improved.

According to a 7th aspect of the present invention, there is provided amedium which contains a program for implementing data processing for abitstream including coded data which are obtained by successively codingimage data corresponding to a moving picture for each of firstprocessing units and synchronization signals each of which is added tothe coded data for each second processing unit including a plurality ofthe first processing units, by a computer, and the data processingcomprises: a decoding process of decoding the coded data included in thebitstream for each of the first processing units and generating decodedimage data; an error detection process of detecting a first erroroccurring during transmission of the bitstream and a second error otherthan the first error separately; and a concealment process of performingconcealment of the decoded image data taking the first processing unitas a concealment unit when the first error is detected, and performingconcealment of the decoded image data taking the second processing unitas a concealment unit when the second error is detected. Therefore, amoving picture decoding method in which the concealment processing forthe decoded image data is performed effectively according to whether theerror is the transmission error or stream error, thereby improving theimage quality of decoded images can be realized by software.

According to an 8th aspect of the present invention, there is provided amedium which contains a program for implementing data processing for abitstream including coded data which are obtained by successively codingimage data corresponding to a moving picture for each of unit regionsconstituting one VOP of the moving picture, by a computer, and the dataprocessing comprises: a decoding process of decoding the coded dataincluded in the bitstream for each of the unit regions and generatingdecoded image data; an error detection process of detecting an errorincluded in the bitstream; a process of judging whether or not thebitstream has shape information indicating a shape of the movingpicture; and a concealment process of performing concealment of thedecoded image data taking one VOP of the moving picture as a concealmentunit when the error in the bitstream is detected and it is judged thatthe bitstream has the shape information, and performing concealment ofthe decoded image data taking a processing region including at least oneof the unit regions, which processing region is smaller than the VOP, asa concealment unit when the error in the bitstream is detected and it isjudged that the bitstream has no shape information. Therefore, a movingpicture decoding method in the deterioration in image quality resultingfrom the error can be excluded without causing high deterioration inimage quality due to concealment of the decoded image when the bitstreamincludes the shape information, and besides the deterioration in imagequality resulting from the error can be excluded by the simpleconcealment processing when the bitstream include no shape informationcan be realized by software.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1( a) and 1(b) are diagrams for explaining a moving picturedecoding apparatus according to a first embodiment of the presentinvention, FIG. 1( a) illustrating the structure of the moving picturedecoding apparatus and FIG. 1( b) illustrating the data structure of aVOP bitstream which is input to the moving picture decoding apparatus.

FIG. 2 is a flowchart showing processing of decoding an input streamusing the moving picture decoding apparatus of the first embodiment.

FIG. 3 is a block diagram for explaining a moving picture decodingapparatus according to a second embodiment of the present invention.

FIG. 4 is a flowchart showing decoding processing by the moving picturedecoding apparatus of the second embodiment.

FIG. 5 is a block diagram for explaining a moving picture decodingapparatus according to a third embodiment of the present invention.

FIG. 6 is a flowchart showing decoding processing by the moving picturedecoding apparatus of the third embodiment.

FIGS. 7( a)-7(c) are diagrams for explaining the case where the movingpicture decoding processing according to any of the embodiments isimplemented using a floppy disk which contains a moving picture decodingprogram by a computer system, FIGS. 7( a) and 7(b) illustrating thefloppy disk and FIG. 7( c) illustrating the computer system.

FIGS. 8( a)-8(c) are diagrams for schematically showing variousprocessing units in coding processing conforming to MPEG-4, FIG. 8( a)illustrating the shape of an object, FIG. 8( b) illustrating the textureof the object, and FIG. 8( c) illustrating a rectangular regioninvolving the object.

FIGS. 9( a)-9(e) are diagrams for schematically showing variousprocessing units for a bitstream conforming to MPEG-4, FIG. 9( a)illustrating a video packet, FIG. 9( b) illustrating a regioncorresponding to the video packet in a VOP, FIG. 9( c) illustrating amacroblock, FIG. 9( d) illustrating a block, and FIG. 9( e) illustratinga variation of the video packet.

FIG. 10 is a diagram for schematically showing coding processing inobject units according to MPEG-4.

FIGS. 11( a)-11(c) are diagrams for schematically showing the structureof a bitstream according to MPEG-4 in detail, FIG. 11( a) illustrating aVOP bitstream, FIG. 11( b) illustrating a video packet, and FIG. 11( c)illustrating macroblock information.

FIG. 12 is a block diagram for explaining a conventional common movingpicture decoding apparatus.

FIG. 13 is a block diagram for explaining a conventional another movingpicture decoding apparatus.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described.

Embodiment 1

FIGS. 1( a) and (b) are diagrams for explaining a moving picturedecoding apparatus according to a first embodiment of the presentinvention. FIG. 1( a) illustrates the structure of the moving picturedecoding apparatus. FIG. 1( b) illustrates the data structure of a VOPbitstream which is input to the moving picture decoding apparatus.

The moving picture decoding apparatus 100 a according to the firstembodiment performs processing of decoding a bitstream which is input asimage coded information to generate decoded image data. In the decodingprocessing, the decoded image data are concealed in macroblock unitswhen a transmission error is detected and the decoded image data areconcealed in video packet units when a stream error other than thetransmission error is detected.

Here, each of VOP bitstreams constituting the bitstream, correspondingto each VOP of one image sequence, is divided in plural data units(video packets) comprising a fixed length code (marker code) of aspecific bit pattern as a synchronization signal and coded informationsubsequent to the marker code. The coded information of each videopacket includes plural pieces of macroblock information as data units,each corresponding to a macroblock. Thus, the VOP bitstream has atwo-layer data structure which is divided by data processing units(second processing units) each corresponding to the video packet, anddata processing units (first processing units) each corresponding to themacroblock.

To be more specific, the moving picture decoding apparatus 100 aaccording to the first embodiment has a decoder 1 for subjecting VOPbitstreams constituting a bitstream Vin which is input as image codedinformation (hereinafter referred to as input bitstream) to decodingprocessing including the variable-length decoding, and a memory 2temporarily containing decoded image data Vout corresponding to analready processed VOP whose decoding processing is finished andoutputting a part of the contained decoded image data as reference imagedata Vref which are referred to during the decoding processing andreplacement image data Vrep which are used for concealment processing.In this case, the decoder 1 and the memory 2 have the structuresidentical to those of the decoder 101 and the memory 102 in theconventional moving picture decoding apparatus 100 and 110.

Further, the moving picture decoding apparatus 100 a has an MB selectorswitch 5 for selecting one of the decoded image data Vd from the decoder1 and the replacement image data Vrep from the memory 2 in accordancewith an MB selection control signal Cmb, and outputting the selectedimage data as MB selected image data Emb; a first VP delay circuit 1 afor delaying the MB selected image data Emb for a time required for thedecoding processing of a target video packet to be processed; a secondVP delay circuit 2 a for delaying the replacement image data Vrep whichare output from the memory 2 in synchronization with the decodingprocessing of each macroblock for a time required for the decodingprocessing of the target video packet; and a VP selector switch 8 forselecting one of the output of the first delay circuit 1 a (MB delayedselected data) DEmb and the output of the second delay circuit 2 a(delayed replacement data) DVrep in accordance with a VP selectioncontrol signal Cvp, and outputting the selected image data as VPselected image data Evp.

Here, the MB selector switch 5 has a first input terminal 5 a to whichthe decoded image data Vd from the decoder 1 are supplied, a secondinput terminal 5 b to which the replacement image data Vrep from thememory 2 are supplied, and an output terminal 5 c for outputting the MBselected image data Emb. The MB selector switch 5 can be switched inaccordance with the MB selection control signal Cmb between the statewhere the first input terminal 5 a is connected to the output terminal 5c and the state where the second input terminal 5 b is connected to theoutput terminal 5 c. The VP selector switch 8 has a first input terminal8 a to which the MB delayed selected data DEmb from the first VP delaycircuit 1 a are supplied, a second input terminal 8 b to which thedelayed replacement data DVrep from the second VP delay circuit 2 a aresupplied, and an output terminal 8 c for outputting the VP selectedimage data Evp. The VP selector switch 8 can be switched in accordancewith the VP selection control signal Cvp between the state where thefirst input terminal 8 a is connected to the output terminal 8 c and thestate where the second input terminal 8 b is connected to the outputterminal 8 c.

Further, this moving picture decoding apparatus 100 a has a transmissionerror detector 3 for detecting transmission errors in the input streamVin and outputting a transmission error notification signal Terr; and astream error detector 6 for detecting stream errors in the input streamVin on the basis of the transmission error notification signal Terr fromthe transmission error detector 3 and an internal signal Si of thedecoder 1, and outputting a stream error notification signal Serr.

The transmission error detector 3 detects the transmission error on thebasis of a marker code indicating the absence of the packet in the inputstream Vin, like the error detector 120 in the conventional movingpicture decoding apparatus 100. The marker code is inserted into theinput stream by an error check unit 14 which is provided in the previousstage of the moving picture decoding apparatus 100 a. The error checkunit 14 specifies the position of a defect caused by the transmissionerror in the input stream on the basis of the analog signal level of theinput stream or error-correcting codes, and inserts the marker code inthis defective position.

To be specific, only when the stream error detector 6 detects thefailure of the decoding processing for the input stream Vin on the basisof the internal signal Si of the decoder 1 and detects that notransmission error occurs on the basis of the transmission errornotification signal Terr, the detector 6 outputs the stream errornotification signal Serr as a signal indicating that the stream error isdetected. The stream error detector 6 can have a structure for strictlyexamining the structure and contents of the bitstream in the videopacket, instead of detecting the failure of the decoding processing, andoutputting the stream error notification signal Serr only when thedetector 6 detects the abnormality of the bit stream and detects that notransmission error occurs.

Further, the moving picture decoding apparatus 100 a has a macroblockunit concealer 4 for controlling the MB selector switch 5 in accordancewith the MB selection control signal Cmb on the basis of thetransmission error notification signal Terr, and replacing a part whichis affected by the transmission error in the decoded image data Vd ofthe target VOP with the decoded image data (replacement image data) Vrepof the already processed VOP in macroblock units; and a video packetunit concealer 7 for controlling the VP selector switch 8 in accordancewith the VP selection control signal Cvp on the basis of the streamerror notification signal Serr, and replacing a part which is affectedby the stream error in the delayed image data DVd of the target VOP withthe delayed replacement data DVrep of the already processed VOP in videopacket units.

The macroblock unit concealer 4 has the same structure as the macroblockunit concealer 104 in the conventional moving picture decoding apparatus100 as shown in FIG. 12.

Next, the operation of the moving picture decoding apparatus 100 a isdescribed.

Initially, a brief explanation is given of the decoding processing bythe moving picture decoding apparatus 100 a according to the firstembodiment.

FIG. 2 is a flowchart showing the decoding processing by the movingpicture decoding apparatus 100 a according to the first embodiment.

When the bitstream (input stream) Vin is input as coded information ofthe image signal to the moving picture decoding apparatus 100 a of thefirst embodiment, the stream of a target VOP is subjected to thedecoding processing with reference to the decoded image data Vref of thealready processed VOP for each macroblock, and the decoded image data Vdcorresponding to each macroblock are generated (step S1 a).

Then, the processing of detecting the transmission errors in the inputstream Vin is performed (step S2 a).

When the transmission error is detected by the transmission errordetection processing, the concealment processing for the decoded imagedata is performed in macroblock units (step S3 a). That is, decodedimage data of an error macroblock which are obtained from the macroblockinformation including the transmission error are replaced with decodedimage data Vrep of a macroblock in the already processed VOP,corresponding to the error macroblock. On the other hand, when notransmission error is detected as a result of the error detectionprocessing, the processing of detecting the stream errors in the inputstream Vin is performed (step S4 a).

Only when the stream error is detected by the stream error detectionprocessing, the concealment processing for the decoded image data isperformed in video packet units. That is, the delayed decoded data DVdcorresponding to the error video packet including the stream error arereplaced with the delayed replacement data DVrep corresponding to theerror video packet (step S5 a).

Hereinafter, the operation of the moving picture decoding apparatus 100a is described in detail.

When the bitstream which is read from a recording medium or thebitstream which is transmitted via a transmission medium is input to themoving picture decoding apparatus 100 a as an input stream Vin, thedecoding processing for the input stream Vin is performed by the decoder1 with reference to the decoded image data Vref of the already processedVOP from the memory 2 in this moving picture decoding apparatus 100 a.At this time, in the transmission error detector 3, the detection of thetransmission error is performed on the basis of the input stream Vin.When the marker code in the input stream is detected, the transmissionerror notification signal Terr is output to the macroblock unitconcealer 4 and the stream error detector 8.

In the macroblock unit concealer 4, the MB selection control signal Cmbis output to the MB selector switch 5 on the basis of the transmissionerror notification signal Terr. Accordingly, in the MB selector switch5, one of the decoded image data Vd of the target VOP from the decoder 1and the decoded image data Vrep of the already processed VOP from thememory 2 is selected for each macroblock, and the selected image dataare output as the MB selected image data Emb.

To be specific, when the transmission error is detected, the decodedimage data Vd corresponding to macroblock information from the headmacroblock information in the VOP bitstream to the macroblockinformation immediately precedent to the error macroblock informationincluding the marker code are selected by the MB selector switch 5, andoutput as the MB selected image data Emb. The decoded image data Vdcorresponding to macroblock information from the error macroblockinformation to macroblock information just before the nextsynchronization signal are not selected by the MB selector switch 5. Thedecoded image data Vrep of the already processed VOP corresponding tothe decoded image data Vd are selected by the MB selector switch 5 asthe replacement image data, and the selected data are output as the MBselected image data Emb.

Hereinafter, the operation of the MB selector switch 5 is described inmore detail with reference to FIG. 1( b). Here, a description is givenof the case where the marker code Cm is included in the i-th macroblockinformation Smb(i) in the k-th video packet Svp(k) in the input VOPbitstream Svop, i.e., the defect due to the transmission error isincluded in the macroblock information Smb(i) in the video packetSvp(k).

In this case, decoded image data corresponding to respective macroblockinformation from the first macroblock information Smb (1) to the(i−1)-th macroblock information Smb(i−1) in the k-th video packet Svp(k)are selected by the MB selector switch 5. Decoded image datacorresponding to macroblock information from the i-th macroblockinformation Smb(i) and thereafter in the k-th video packet Svp(k) arenot selected by the MB selector switch 5. In the MB selector switch 5,the decoded image data Vrep of the macroblock in the already processedVOP, corresponding to the decoded image data Vd of these macroblocks areselected as the replacement image data. In this case, the (i+1)-thmacroblock information in the k-th video packet Svp(k) is missing due tothe transmission error. Further, the (i+2)-th, (i+3)-th, . . . , (n)-thmacroblock information Smb(i+2), Smb(i+3), . . . , Smb (n) in the k-thvideo packet Svp(k) are the (i+2)-th, (i+3)-th, (n)-th macroblockinformation of the k-th video packet Svp(k) respectively. Here, themacroblock information Smb(i+2)HSmb(n) are information which have beenreceived without transmission errors. However, when the (i+1)-thmacroblock information cannot be decoded correctly, the (i+2)-th,(i+3)-th, . . . , (n)-th macroblock information cannot be decodedcorrectly either. Therefore, decoded image data Vrep corresponding toall of the (i+1)-th to (n)-th macroblock information are replaced withreplacement image data.

In FIG. 1( b), Svoph is the header of the VOP bitstream and includes thesynchronization signal. In addition, Svp(1), Svp(k+1) and Svp(m) are thefirst, the (k+1)-th and the last video packets, constituting the VOPbitstream Svoph, respectively. Svph(1), Svph(k), Svph(k+1) and Svph(m)are the headers of the video packets Svp(1), Svp(k), Svp(k+1) andSvp(m), respectively.

In the stream error detector 6, the judgement as to whether or not thetransmission error is included in the input stream is made in accordancewith the transmission error notification signal Terr from thetransmission error detector 3, and the judgement as to whether or notthe decoding processing for the input stream Vin fails is made inaccordance with the internal signal Si of the decoder 1. Then, theprocessing of detecting the stream errors in the input stream Vin isperformed on the basis of these judgements.

For example, when the decoding processing for the input stream fails,the stream error detector 6 detects the failure of the decodingprocessing in accordance with the internal signal Si of the decoder 1.At this time, when the detection of the transmission error is notifiedby the transmission error notification signal Terr, the failure of thedecoding processing is caused by the transmission error. Therefore, inthis case, the stream error notification signal Serr is not output. Onthe other hand, when there is no notification of occurrence of thetransmission error by the transmission error notification signal Terr,the failure of the decoding processing is caused by the stream error.Therefore, in this case, it is judged that the stream error is includedin the input stream, and the stream error notification signal Serr isoutput to the video packet unit concealer 7.

Here, in the stream error detector 6, it is possible that the detectionprocessing of the stream error is not performed for the video packetincluding the transmission error in the state where the transmissionerror detector 3 detects the transmission error.

In the video packet unit concealer 7, the VP selection control signalCvp is output to the VP selector switch 8 in accordance with the streamerror notification signal Serr. Thus, in the VP selector switch 8, oneof the delayed selected data DEmb of the target VOP from the first VPdelay circuit 1 a and the delayed replacement data DVrep of the alreadyprocessed VOP from the second VP delay circuit 2 a is selected for eachvideo packet, and the VP selected image data Evp are output as thereproduced image data Vout from the VP selector switch 8.

To be specific, when the stream error is included in the input streamVin, the delayed decoded data DVd corresponding to the error videopacket in which the decoding processing in the VOP bitstream fails arenot selected by the VP selector switch 8 but the delayed replacementdata (decoded image data of the video packet in the already processedVOP) DVrep corresponding to the delayed decoded data DVd are selected bythe VP selector switch 8, and the selected image data are output as theVP selected image data Evp. On the other hand, the delayed decoded dataDVd corresponding to video packets other than the video packet in whichthe decoding processing fails in the VOP stream are selected by the VPselector switch 8 and the selected image data are output as the VPselected image data Evp.

The VP selected image data Evp are output as the reproduced image dataVout as well as stored in the memory 2 as the reference image data forthe following VOP subsequent to the target VOP.

Thus, in the first embodiment, the decoder 1 for decoding the inputstream for each macroblock and generating the decoded image data, thetransmission error detector 3 for detecting the transmission errorsincluded in the input stream, and the stream error detector 6 fordetecting the stream errors included in the input stream are provided.Thereby, the decoded image data of a macroblock which is affected by thetransmission error are replaced with the decoded image data of thecorresponding macroblock in the already processed VOP, and the decodedimage data corresponding to the video packet including the stream errorare replaced with the decoded image data of the corresponding videopacket of the already processed VOP. Therefore, the concealment for thetransmission error of the decoded image data is performed in macroblockunits and the concealment for the stream error of the decoded image datais performed in video packet units.

Accordingly, when the transmission error occurs, only the decoded imagedata of the macroblock which is affected by the transmission error areconcealed. Therefore, the deterioration in image quality of the decodedimage due to the concealment of the decoded image data of macroblockswhich are not affected by the transmission error can be avoided. Inaddition, when the stream error occurs, the decoded image data of allmacroblocks corresponding to the video packet including the stream errorare concealed. Therefore, the deterioration in image quality of thedecoded image due to the output of the decoded image data of themacroblock which is affected by the stream error can be prevented.Consequently, the concealment processing of the decoded image data forthe errors included in the input stream is effectively performed,thereby improving the image quality of the decoded image.

In this first embodiment, the description is given of the case where theinput stream has a two-layer data structure, i.e., the case where theVOP bitstreams constituting the input stream are divided in the videopacket units and further the video packet is divided in units ofmacroblocks as coding processing units. However, the two-layer datastructure of the input stream is not restricted to that as shown in thefirst embodiment.

For example, the input stream can have a data structure in which the VOPbitstream is divided in units of video packets and the video packet isdivided in units of blocks (8×8 pixels) as minimum units of the codingprocessing, not in macroblock units. In this case, the concealment forthe transmission error of the decoded image data is performed in blockunits, whereby the same effects as those in the first embodiment can beobtained.

The input stream can have any two-layer data structure as long as theVOP bitstream corresponding to one VOP of the image sequence is dividedin units of data each comprising a synchronization signal and codedinformation subsequent thereto, and this data unit is divided so as tocorrespond to regions dividing the VOP, as units of the codingprocessing. In this case, the concealment for the transmission error ofthe decoded image data is performed in data units (first processingunits) corresponding to units of the coding processing, and theconcealment for the stream error of the decoded image data is performedin data units (second processing units) each including thesynchronization signal, whereby the same effects as those in the firstembodiment can be obtained.

Further, in this first embodiment, the input stream having the two-layerdata structure is described. However, the input stream can have a datastructure of three or more layers. The input stream can have anymulti-layer data structure, as long as the input stream is divided intodata units (second processing units) each comprising the synchronizationsignal and coded information subsequent thereto and this data unit isdivided so as to correspond to data units (first processing units)corresponding to the unit of the coding processing.

In this first embodiment, the macroblock corresponding to the data unit(first processing unit) which divides the video packet is an image spacecomposed of 16×16 pixels.

However, the number of pixels constituting the macroblock in thehorizontal and vertical directions can be varied according to themethods for coding the moving pictures.

Embodiment 2

FIG. 3 is a block diagram for explaining a moving picture decodingapparatus according to a second embodiment of the present invention.

The moving picture decoding apparatus 100 b according to the secondembodiment performs concealment of decoded image data in macroblockunits when a transmission error is detected and performs concealment ofthe decoded image data when a stream error is detected in video packetunits, as in the first embodiment, in decoding processing of a bitstreamcorresponding to an object having no shape, and conceals the decodedimage data in VOP units when the error is detected in the decodingprocessing of the bitstream corresponding to an object having the shape.

This moving picture decoding apparatus 100 b of the second embodimenthas a circuit structure for performing the concealment of decoded imagedata in VOP units, in addition to the structure of the moving picturedecoding apparatus 100 a of the first embodiment.

To be specific, this moving picture decoding apparatus 100 b of thesecond embodiment has the decoder 1, the memory 2, the detectors 3 and6, the concealers 4 and 7, the selector switches 5 and 8, and the delaycircuits 1 a and 2 a, as in the above-mentioned first embodiment.

This moving picture decoding apparatus 100 b of the second embodimenthas an OR circuit 9 for outputting an error detection signal TSerrindicating that the input stream includes transmission errors or streamerrors, on the basis of the OR operation of the transmission errornotification signal Terr from the transmission error detector 3 and thestream error notification signal Serr from the stream error detector 6;and a unit for detecting the presence or absence of the shape(hereinafter referred to as presence-or-absence-of-shape detector) 10for judging whether or not the input stream Vin has the shapeinformation on the basis of the input stream and outputting a signal fornotifying the presence or absence of the shape (hereinafter referred toas presence-or-absence-of-shape-notification signal) Sdet according tothe judgement result.

In addition, the moving picture decoding apparatus 100 b has a first VOPdelay circuit 1 b for delaying the VP selected image data Evp which areoutput by the VP selector switch 8 for a time required for the decodingprocessing of a target VOP to be processed and outputting VOP delayedselected data DEvp; and a second VOP delay circuit 2 b for delaying theVP delayed replacement data (first delayed replacement data) DVrep1which are output by the second VP delay circuit 2 a for a time requiredfor the decoding processing of one VOP and outputting VOP delayedreplacement data (second delayed replacement data) DVrep2.

Further, the moving picture decoding apparatus 100 b has a VOP selectorswitch 13 for selecting one of the VOP delayed selected data DEvp fromthe first VOP delay circuit 1 b and the VOP delayed replacement dataDVrep2 from the second VOP delay circuit 2 b in accordance with a VOPselection control signal Cvop, and outputting the selected image data asVOP selected image data Evop; an AND circuit 11 for outputting streaminformation Istr indicating that the input stream having the shapeinformation includes an error on the basis of the AND operation of theerror detection signal TSerr and thepresence-or-absence-of-shape-notification signal Sdet; and a VOP unitconcealer 12 for controlling the VOP selector switch 13 so that the VOPdelayed selected data DEvp from the first VOP delay circuit 1 b areconcealed in VOP units when the error included in the input streamhaving the shape information is detected on the basis of the streaminformation Istr.

The VOP selector switch 13 has a first input terminal 13 a to which theVOP delayed selected data DEvp from the first VOP delay circuit 1 b aresupplied, a second input terminal 13 b to which the VOP delayedreplacement data DVrep2 from the second VOP delay circuit 2 b aresupplied, and an output terminal 13 c for outputting the VOP selectedimage data Evop. The VOP selector switch 13 can be switched inaccordance with the VOP selection control signal Cvop between the statewhere the first input terminal 13 a is connected to the output terminal13 c and the state where the second input terminal 13 b is connected tothe output terminal 13 c.

Next, the effects are described.

Initially, a description is given of the fundamental principles of thesecond embodiment.

When decoded image data having shape information of an object areconcealed in macroblock units or video packet units, the continuity ofthe object shape in the VOP is usually harmed. Thus, in the case wherethe bitstream has the shape information, visually more preferabledecoded images are often obtained when decoded image data correspondingto bitstreams including errors are concealed in VOP units, as comparedto when the decoded image data are concealed in macroblock units orvideo packet units.

On the other hand, when the image has no shape information, there aremany cases where temporal changes in the shape are small and thecorrelation of pixel values between VOPs is strong. Therefore, it ispreferable that the concealment of the decoded image data is performedin macroblock units or video packet units using the decoded image dataof the already processed VOP.

Thus, in this second embodiment, it is judged whether or not the inputstream Vin has shape information on the basis of the input stream. Whenthe input stream has the shape information, the decoded image data aresubjected to the concealment processing in VOP units. When the inputstream has no shape information, the decoded image data are subjected tothe concealment processing in macroblock units or video packet units.

Initially, a brief explanation is given of the decoding processing bythe moving picture decoding apparatus 100 b of the second embodiment.

FIG. 4 is a flowchart showing the decoding processing by the movingpicture decoding apparatus 100 b of the second embodiment.

When a bitstream including coded information corresponding to a movingpicture is input as the input stream Vin to the moving picture decodingapparatus 100 b of the second embodiment, the processing of decoding thecoded information corresponding to a target VOP to be processed in theinput stream (target VOP bitstream to be processed) for each macroblockis performed successively in the decoder 1 (step S1 b).

Then, it is judged by the presence-or-absence-of-shape detector 10whether or not the input stream Vin includes a shape signal. As a resultof the judgement, when the input stream Vin is a bitstream including noshape information, the same processes as those in steps S2 a to S5 a inthe first embodiment are performed in the respectively correspondingsteps Sib to S6 b. On the other hand, when the input stream is abitstream including the shape information, the judgement as to whetheror not the input stream Vin includes transmission errors (step S7 b) andthe judgement as to whether or not the input stream Vin includes streamerrors (step S8 b) are made.

When the input stream Vin includes either the transmission errors orstream errors, the processing of concealing the decoded image data whichare obtained by the decoding processing for the input stream Vin in VOPunits is performed (step S9 b).

To be specific, when the target VOP bitstream includes an error, thedecoded image data DEvp which are obtained by the decoding processingfor the target VOP bitstream are replaced with the decoded image dataDVrep2 corresponding to the already processed VOP. The decoded imagedata DVrep2 are output as the reproduced image data Vout from the movingpicture decoding apparatus 100 b.

On the other hand, when no error is detected in the input stream Vin asa result of the judgements in steps S7 b and S8 b, the decode image dataVd which are obtained by the decoding processing for the target VOPbitstream are output as the reproduced image data Vout from the movingpicture decoding apparatus 100 b.

Hereinafter, the operation of the moving picture decoding apparatus 100b is described in detail.

When the same input stream Vin as in the moving picture decodingapparatus 100 a of the first embodiment is input to the moving picturedecoding apparatus 100 b of the second embodiment, the decodingprocessing for the input stream Vin by the decoder 1, the detection ofthe transmission errors by the error detector 3, and the detection ofthe stream errors by the error detector 6 is performed in the movingpicture decoding apparatus 100 b.

At this time, in the decoder 1, the decoding processing for the inputstream Vin is performed for each macroblock and the decoded image dataVd are output for each macroblock. From the memory 2, the decoded imagedata (replacement image data) Vrep of a macroblock in the alreadyprocessed VOP, corresponding to the macroblock in the target VOP to beprocessed by the decoder 1 are output in synchronization with thedecoded image data Vd of each macroblock in the target VOP. Further, thetransmission error notification signal Terr from the error detector 3 isinput to the macroblock unit concealer 4 and the stream errornotification signal Serr from the error detector 6 is output to thevideo packet unit concealer 6. The transmission error notificationsignal Terr and the stream error notification signal Serr are output tothe OR circuit 9.

In the presence-or-absence-of-shape detector 10, it is judged on thebasis of the input stream Vin whether or not the input stream Vinincludes the shape information, and thepresence-or-absence-of-shape-notification signal Sdet indicating thejudgement result is output to the AND circuit 11.

Usually, in a VOP bitstream including shape information, informationsuch as flag information indicating that shape information is includedin its sequence header or the like is added. Therefore, the judgement asto whether or not the shape information is included is made on the basisof this flag information.

Then, the control for switching the MB selector switch 5 is performed bythe macroblock unit concealer 4 and the control for switching the VPselector switch 8 is performed by the video packet unit concealer 6. Inthe OR circuit 9, the error notification signal TSerr indicating thatthe input stream includes one of the transmission error or stream erroris output to the AND circuit 11 according to the OR operation of thetransmission error notification signal Terr and the stream errornotification signal Serr.

In this AND circuit 11, the stream information Istr indicating that theinput stream is a bitstream having the shape information and includes anerror is output to the VOP unit concealer 12 according to the ANDoperation of the error notification signal TSerr and thepresence-or-absence-of-shape-notification signal Sdet.

In the MB selector switch 5, one of the decoded image data Vd and thereplacement image data Vrep is selected in accordance with the MBselection control signal Cmb from the macroblock unit concealer 4, andthe selected image data are output as the MB selected image data Emb.

The MB selected image data Emb from the MB selector switch 5 and thereplacement image data Vrep from the memory 2 are delayed for a timerequired for the decoding processing for the target video packet by thefirst and second VP delay circuits 1 a and 2 a, and output as the VPdelayed selected data DEmb and the VP delayed replacement data (firstdelayed replacement data) DVref1, respectively.

Further, in the VP selector switch 8, one of the VP delayed selecteddata DEmb and the VP delayed replacement data DVrep1 is selected inaccordance with the VP selection control signal Cvp from the videopacket unit concealer 7, and the selected delayed data are output as theVP selected image data Evp.

The VP selected image data Evp from the VP selector switch 8 and the VPdelayed replacement data DVrep1 from the second VP delay circuit 2 a aredelayed for a time required for the decoding processing for the targetVOP by the first and second VOP unit delay circuits 1 b and 2 b, andoutput as the VOP delayed selected data DEvp and the VOP delayedreplacement data (second delayed replacement data) DVrep2, respectively.

In the VOP selector switch 13, one of the VOP delayed selected data DEvpand the VOP delayed replacement data DVrep2 is selected in accordancewith the VOP selection control signal Cvop from the VOP unit concealer12, and the selected delayed data are output as VOP selected image dataEvop. The VOP selected image data Evop are stored in the memory 2 aswell as output as the reproduced image data Vout.

To be specific, the VOP selector switch 13 is controlled by the VOP unitconcealer 12 so that the VP delayed selected data Evp from the first VOPdelay circuit 1 b are output as they are when the input stream includesno shape information or the input stream includes no error, and the VPdelayed selected data Evp from the first VOP delay circuit 1 b arereplaced with the VOP delayed replacement data DVrep2 from the secondVOP delay circuit 2 b when the input stream has the shape informationand includes the error.

Thus, in the second embodiment, the presence-or-absence-of-shapedetector 10 for judging whether or not the input stream Vin has theshape information is provided in addition to the structure of the firstembodiment. In the case where the input stream Vin has the shapeinformation, when the input stream Vin includes the transmission erroror stream error the decoded image data which are obtained by thedecoding processing of the input stream Vin are concealed in VOP units.On the other hand, in the case where the input stream Vin has no shapeinformation, the decoded image data are concealed in macroblock units orvideo packet units according to the type of the error included in theinput stream Vin in the same way as in the above-mentioned firstembodiment. Therefore, in addition to the effects of the firstembodiment, the deterioration of image quality caused by errors in thedecoded image which is obtained from the input stream Vin having theshape information can be excluded with suppressing the deterioration inimage quality resulting from the concealment of the decoded image data.

In the first and second embodiments, the transmission error detector 3detects the transmission error by detecting a mark (marker code)inserted in the bitstream, which indicates the absence of the packet.However, the transmission error detector can have a structure forobtaining information relating to the transmission error occurrenceposition in the input stream from the transmission system in another wayand outputting the transmission error notification signal Terr.

Embodiment 3

FIG. 5 is a block diagram for explaining a moving picture decodingapparatus according to a third embodiment.

This moving picture decoding apparatus 100 c of the third embodimentconceals decoded image data in video packet units when an error in aninput stream is detected in decoding processing for a bitstreamcorresponding to an object having no shape, and conceals the decodedimage data in VOP units when the error in the input stream is detectedin the decoding processing f or a bitstream corresponding to an objecthaving the shape.

The moving picture decoding apparatus 100 c of the third embodiment hasthe decoder 1, the memory 2, the concealers 7 and 12, the delay circuits1 a, 2 a, 1 b and 2 b, the selector switches 8 and 13, the AND circuit11, and the presence-or-absence-of-shape detector 10, like the movingpicture decoding apparatus 100 b in the second embodiment.

Then, this moving picture decoding apparatus 100 c has an error detector18, in place of the transmission error detector 3, the stream errordetector 6 and the OR circuit 9 in the moving picture decoding apparatus100 b in the second embodiment, for performing processing of detectingtransmission errors and stream errors on the basis of the input streamVin and the internal signal Si of the decoder 1, and outputting an errornotification signal Aerr when one of the errors is detected.

Further, in this moving picture decoding apparatus 100 c, the errornotification signal Aerr is input to the video packet unit concealer 7,and the error notification signal Aerr and thepresence-or-absence-of-shape-notification signal Sdet from thepresence-or-absence-of-shape detector 10 are input to the AND circuit11.

Further, in this moving picture decoding apparatus 100 c, the macroblockunit concealer 4 and the MB selector switch 5 in the moving picturedecoding apparatus 100 b of the second embodiment are omitted. Theoutput Vd of the decoder 1 is directly input to the first VP delaycircuit 1 a.

Next, the effects of the moving picture decoding apparatus 100 c aredescribed.

According to the second embodiment, in the decoding processing for thebitstream corresponding to an object having no shape, the concealmentprocessing when the transmission error is detected is performed inmacroblock units and the concealment processing when the stream error isdetected is performed in video packet units. However, when the number ofmacroblocks corresponding to the video packets is less, i.e., when thenumber of pieces of the macroblock information included in the videopacket is less, the deterioration in image quality resulting from theconcealment processing for the decoded image is lower even when theconcealment processing for the decoded image data having no shapeinformation is always performed in video packet units.

In addition, when the concealment processing for the decoded image datain macroblock units is omitted, the concealment processing can besimplified.

Thus, in the third embodiment, unlike the second embodiment, when theinput stream has no shape information, the concealment processing forthe decoded image is performed in video packet units in either casewhere the transmission error or the stream error is detected as theerror in the input stream.

Initially, a brief explanation is given of the decoding processing bythe moving picture decoding apparatus 100 c of the third embodiment.

FIG. 6 is a flowchart showing the decoding processing by the movingpicture decoding apparatus 100 c of the third embodiment.

When the bitstream including coded information corresponding to a movingpicture is input to the moving picture decoding apparatus 100 c as theinput stream Vin, processing of decoding a part (target VOP bitstream tobe processed) corresponding to a target VOP to be processed in the inputstream for each macroblock is performed successively in the decoder 1(step S1 c).

Then, it is judged by the presence-or-absence-of-shape detector 10whether or not the input stream Vin includes a shape signal (step S2 c).

As a result of the judgement, when the input stream Vin is a bitstreamincluding no shape information, the processing of detecting errors inthe input stream Vin is performed by the error detector 18 (step S3 c).Then, when the error in the input stream Vin is detected, decoded imagedata which are obtained by the decoding processing for the input streamVin are concealed in video packet units (step S4 c).

On the other hand, as a result of the judgement in step S2 c, when theinput stream is a bitstream including the shape information, theprocessing of detecting the transmission error in the input stream Vinis performed by the error detector 18 (step S5 c). Then, when the errorin the input stream Vin is detected, the decoded image data which areobtained by the decoding processing for the input stream Vin areconcealed in VOP units (step S6 c).

As a result of the judgements in steps S3 c and S6 c, when no error isdetected in the input stream Vin, the decoded image data Vd which areobtained by the decoding processing for the target VOP stream are outputas reproduced image data Vout from the moving picture decoding apparatus100 c.

Hereinafter, the operation of the moving picture decoding apparatus 100c is described in detail.

When the same input stream Vin as in the moving picture decodingapparatus 100 a of the first embodiment is input to the moving picturedecoding apparatus 100 c of the third embodiment, the processing ofdetecting whether or not the input stream Vin includes the shapeinformation is performed by the presence-or-absence-of-shape detector 10in this moving picture decoding apparatus 100 c as in the secondembodiment, and the presence-or-absence-of-shape-notification signalSdet is output. In the decoder 1 of the moving picture decodingapparatus 100 c, the decoding processing for the input stream Vin isperformed and the decoded image data corresponding to the target VOP areoutput for each macroblock, as in the second embodiment. At this time,from the memory 2, the decoded image data (replacement image data) Vrepof a macroblock in the already processed VOP, corresponding to amacroblock in the target VOP to be processed by the decoder 1 are outputin synchronization with the decoded image data Vd of each macroblock inthe target VOP.

In the error detector 18 of the third embodiment, the processing ofdetecting errors in the input stream Vin is performed on the basis ofthe input stream Vin and the internal signal Si of the decoder 1. Whenthe error is detected, the error notification signal Aerr is output tothe video packet unit concealer 7 and the AND circuit 11. In this ANDcircuit 11, the AND operation of the error notification signal Aerr andthe presence-or-absence-of-shape-notification signal Sdet is executedand the stream information Istr indicating that the input stream is abitstream having the shape information and includes the error is outputto the VOP unit concealer 12.

In addition, the decoded image data Vd from the decoder 1 and thereplacement image data Vrep from the memory 2 are delayed by the firstand second VP unit delay circuits 1 a and 2 a for a time required forthe decoding processing for the target video packet, and output as theVP delayed decoded data DVd and the VP delayed replacement data DVrep1,respectively.

In the VP selector switch 8, one of the VP delayed decoded data DVd andthe VP delayed replacement data DVrep1 is selected in accordance withthe VP selection control signal Cvp from the video packet unit concealer7, and the selected delayed data are output as the VP selection imagedata Evp.

Further, the VP selected image data Evp from the VP selector switch 8and the VP delayed replacement data Vrep1 from the second VP delaycircuit 2 a are delayed by the first and second VOP delay circuits 1 band 2 b for a time required for the decoding processing for the targetVOP, and output as the VOP delayed selected data DEvp and the VOPdelayed replacement data DVrep2, respectively.

Then, in the VOP selector switch 13, one of the VOP delayed selecteddata DEvp and the VOP delayed replacement data DVrep2 is selected inaccordance with the VOP selection control signal Cvop from the VOP unitconcealer 12, and the selected delayed data are output as the VOPdelayed selected data Evop. The VOP delayed selected data Evop arestored in the memory 2 as well as output as the reproduced image dataVout.

Here, the VOP selector switch is controlled by the VOP unit concealer 12so that the VOP delayed selected data DEvp from the first VOP delaycircuit 1 b are output as they are when it is judged that the inputstream has no shape information or no error is detected in the inputstream, and the VOP delayed selected data DEvp from the first VOP delaycircuit 1 b are replaced with the VOP delayed replacement data DVrep2from the second VOP delay circuit 2 b when the it is judged that theinput stream has the shape information and the error is detected in theinput stream.

Thus, in this third embodiment, the decoded image data which areobtained by the decoding processing for the input stream Vin includingthe shape information are subjected to the concealment processing forthe decoded image in VOP units, and the decoded image data which areobtained by the decoding processing for the input stream Vin includingno shape information are subjected to the concealment processing for thedecoded image in video packet units. Therefore, when the input streamVin include the shape information, the deterioration in image qualitydue to the error can be excluded without high deterioration in imagequality due to the concealment processing for the decoded image.Moreover, when the input stream Vin includes no shape information, thedeterioration in image quality due to the error can be excluded by thesimple concealment processing.

In this third embodiment, when the input stream Vin includes no shapeinformation, the concealment of the decoded image is performed in videopacket units. However, the concealment processing for the decoded imagewhen the input stream Vin includes no shape information can be performedin macroblock units.

In this third embodiment, the error detector 18 has the structure fordetecting both of the transmission error and the stream error. However,the error detector may have a structure for detecting only one of thetransmission error and the stream error.

Further, in the second and third embodiments, the image concealment inVOP units is performed after the image concealment in macroblock unitsor the image concealment in video packet units is performed. However,the image concealment in VOP units may be performed before the imageconcealment in macroblock units and the image concealment in videopacket units.

Further, in the aforementioned embodiments, as the concrete concealmentprocessing for the decoded image, the processing of replacing thedecoded image data corresponding to a target VOP to be decoded with thedecoded image data of the already processed VOP for which the decodingprocessing is finished before the target VOP is shown. However, theconcealment processing for the decoded image is not restricted to thosein the respective embodiments.

For example, the concealment processing for the decoded image can beperformed by replacing the decoded image data of the target VOP withreference image data which are obtained by subjecting decoded image dataof an already processed VOP to the motion compensation processing.Further, the concealment processing for the decoded image can beperformed by subjecting the decoded image data of the target VOP to theintra-frame interpolation processing.

Further, in the aforementioned embodiments, the coding processing anddecoding processing conforming to MPEG-4 standard is shown. However, thecoding processing and decoding processing can be the ones conforming tostandards other than MPEG-4 standard.

Further, when a moving picture decoding program for implementing themoving picture decoding processing according to any of theaforementioned embodiments by a computer is recorded in a data storagemedium such as a floppy disk, the moving picture decoding processingaccording to any of the aforementioned embodiments can be easilyimplemented in an independent computer system.

FIGS. 7( a)-7(b) are diagrams for explaining the case where the movingpicture decoding processing according to any of the aforementionedembodiments is executed by a computer system using a floppy disk whichcontains the moving picture decoding program.

FIG. 7( a) shows a front view of the floppy disk FD, a cross-sectionalview thereof, and a floppy disk body D. FIG. 7( b) shows an example of aphysical format of the floppy disk body D.

The floppy disk FD is composed of the floppy disk body D and a case FCwhich contains the floppy disk body D. On the surface of the floppy diskbody D, plural tracks Tr are formed concentrically from the outercircumference of the disk toward the inner circumference. Each track isdivided into 16 sectors Se in the angular direction. Therefore, in thefloppy disc FD containing the above-mentioned program, data of theprogram are recorded in the assigned sectors on the floppy disk body D.

FIG. 7( c) shows the system structure for recording the moving picturedecoding program in the floppy disk FD and performing the moving picturedecoding processing with software by using the moving picture decodingprogram stored in the floppy disk FD.

When the moving picture decoding program is recorded in the floppy diskFD, data of the moving picture decoding program are written in thefloppy disk FD from the computer system Cs via the floppy disk driveFDD. In addition, when the above-mentioned moving picture decodingapparatus is constructed in the computer system Cs by the programrecorded in the floppy disk FD, the program is read from the floppy diskFD by the floppy disk drive FDD and then loaded to the computer systemCs.

Although in FIG. 7 a floppy disk is employed as a medium for recordingthe program, an optical disk may be employed as the program recordingmedium. Also in this case, the moving picture decoding processing can beperformed by software in the same manner as in the case of the floppydisk. Further, the data storage medium is not restricted to the opticaldisk and the floppy disk, and any medium may be employed as long as itcan contain the program, for example, an IC card or a ROM cassette. Alsoin the case of using these data storage media, the moving picturedecoding processing according to any of the aforementioned embodimentscan be implemented by software as in the case of using the floppy diskor the like.

1. A moving picture decoding apparatus for subjecting a bitstreamincluding coded data which are obtained by successively coding imagedata corresponding to a moving picture, to decoding processing ofdecoding the coded data and generating decoded image data, saidapparatus including: a decoding unit for decoding the coded dataincluded in the bitstream and generating decoded image data; a memoryunit for temporarily storing an image data that is temporarily decoded;and a replacement means for, when an error is detected at decoding thebitstream, changing a replacement unit by which the image data that isobtained by decoding the bitstream is to be replaced with the image datastored in the memory unit, according to whether the error is a firsterror that occurs during transmission of the bitstream or a second errorwhich is detected at decoding the bitstream.
 2. The moving picturedecoding apparatus of claim 1 wherein the image data stored in thememory unit is decoded image data decoding processing for which hasalready been completed.
 3. The moving picture decoding apparatus ofclaim 1 wherein, said replacement means replaces, when the second erroris detected in decoding the bitstream, the data in which the seconderror is detected at decoding with the image data stored in the memoryunit in a second processing unit which includes a plurality of firstprocessing units so as to prevent propagation of the second error, andreplaces, when the first error is included in the bitstream, the dataincluding the first error with the image data stored in the memory unit,for each the first processing units.
 4. The moving picture decodingapparatus of claim 3 wherein the image data stored in the memory unit isdecoded image data decoding processing for which has already beencompleted.
 5. The moving picture decoding apparatus of claim 3 whereineach of said first processing units is a unit by which decoding of theimage data is performed.
 6. The moving picture decoding apparatus ofclaim 3 wherein said second processing unit is a unit which is dividedby predetermined fixed-length codes so as to prevent propagation oferror.
 7. The moving picture decoding apparatus of claim 6 wherein saidpredetermined fixed-length codes are synchronization signals.
 8. Themoving picture decoding apparatus of claim 1 wherein said first error isan error an occurrence position of which can be specified.
 9. The movingpicture decoding apparatus of claim 1 wherein said first error is anerror that is caused by a defect of the bitstream.
 10. The movingpicture decoding apparatus of claim 1 wherein said second error is anerror an occurrence position of which cannot be specified.
 11. Themoving picture decoding apparatus of claim 1 wherein said second erroris an error which is detected in decoding the bitstream, other thanerrors caused by a defect of the bitstream.
 12. The moving picturedecoding apparatus of claim 1 wherein the second error is an error whichis detected in decoding the bitstream when the first error has notoccurred.